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Data from intervieva vith elementary school 
principals are esasined for insights into the' impact of 
a d mini strati ve style on evaluation utilization. Characteristics Df 23 
principals* administrative styles are classified as being either 
"compliance" (CO) or "non-coftpliance" (NCO) oriented^ CO principals 
are characterized by the follovingi (1)^ focuses on implamenting the 
policies cf the Board of Education and on complyj.ng vith federal 
guidelinesi (2) has trouble identifying decisions made at the school 
site; (3) gives as reasons for decisions that they are program 
mandated; and (U) focuses on budgetary matters and other 
externally-monitored variables as factors leading to the decision * 
NCO principals are characterized by these factors; (1) vievs the 
school as an iutonomous unit, vith its ovn capabilities for declslDn 
making and problem solving; (2) readily identifies decisious that 
have been made locally; (t) gives internally generated dati- based 
reasons for decisions; and (4) focuses' on staff input as key decision 
variables, CO principals focus on the utilization of externally 
generated data, e«g^ standardized test data, for evaluation 
acti^vities^ whereas NCO principals focus on the utilization of 
internally generated data from staff meetings, needs assessment, 
criterion referenced tests, etc* (EL) 
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Introduction and Rationale 

Why study the local us 3 of evaluation in decisiori making it schools 
with specially funded progr^s? As citizens and aducators we are 
concarned that the best possible programs are devsloped with public 
funds, and it only seems fair that a certain accountability 
accompany tha use of these^ allocations , As program evaluators , 
it goes without .saying, we are committed to the belief that the 
utilization of . educational program evaluation will result in the 
improvement of educational programs* If evaluation is not used 
at the local level where program improvement for a given population 
can occur directly, then we need to come to an understanding of 
the factors affecting the ^se or lack of use of evaluation data 
for making the decisions that will lead to progrmi Improvement, 

Evaluation asks the questions- V^o? What? When? Under what 
conditions? As demonstrated by what? Today we are asking those 
same questions about the utilization of evaluation. This paper 
focuses on the "who" of evaluation utilization. Who uses evaluation? 
That is, §re there certain characteristics of decision-makers 
themselves --the way they see their r6le--that are associated with 
their use of evaluation information? If so, what are these features? 
How do they interact with the kind of decisions made and- the kind 
of data used? ' 

This Study ... 

At the school level the ultimate decision maker is the principal, 
though there are, of course, many other factors which to var.ious " 



degrees influence the decision he or she makes* The way this process 
works is seen in the following example from the study. 



A school gets some extra money based on need. These 
supplemental district fundrs are awarded according to the 
proportiorr of disadvantaged students at the school. This 
event calls for a decision as to how the funds should be 
used. The staff recognizes that some of the students need 
to improve their attitudes towards school, saying, "We've 
been wanting to do something about this for a long time." 
Additionally, the staff knows there are always _ students in 
need of reinforcement in basic skills. The principal 
has an idea based on his experience at a previous school: 
Why not have af ter-school special interest clubs to improve 
student attitudes and tutoring for those In need of basic 
skill reinforcement? The principal presents his idea in 
a staff meeting, and the staff agrees. Each staff member ^ 
gets to choose the activity in which he or she will participate. 



This example illustrates the various components of a typical dacision 
making context at the school level:' (1), a 'significant occurrence 
(extra district funds) ; (2) a principal Cdecision maker) , O) data 
(staff-perceived needs of students and the past experience of the 
principal); arid (4) the decision (to hold after-school^ special Inter- 
est clubs and basic skills tutoring). A simple model portrays the 
relationship. , . 



Significant 
Occurrence 



C. 

Decision 
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In this particular case, the school's receiving extra funds 
is a significant occurrencei A., which calls for a scnool level 
decision. A decision is made by* the principal which is based 
on hifS.,past experience and the staff -perceivad needs of 
students. 

Although a written needs assessment soliciting input from staff 
and parents had been conducted the previous spring, the principal 
did not mention the resulting data as a decision factor. The 
staff did have input in how the decision would be implemented --^ 
each chose the activity in which he or she would participate-- 
but this input came after ^the original decision was made. The ^ 
basic idea for action was the principal's, 

# 

In terms of the model presented above, we wondered if the 
decision making process sometimes started with the data itself 
(point B. in the model). Looking through our. data we found this 
ex^ple: on the basts of a needs assessment conducted the previous ^ 
spring, a decision was made to improve the school climate. On 
further examination of this case we observed that this decision 
context ,had all the elements of the first example except a 
slgnificaTit occurrence. Instead, the decision evolved directly 
from the data, though the principal was again instrumental in the • 
decision that was reached. The following quote illustrates the 
role that needs assessment data played as a catalyst for the deeisioh. 



"The place where ^J^^s^s assessment/ is useful is the 
information and Insight that It gives us . . . . For 
example, at fihe end of last year It was pretty ^lear 
that we had a variety of problems relating to the ^ 
school climate that we had to deal with.' (13P) 

The principal then called a staff meeting for the purpose of 

identifying the focus for the next year's school-level plan. 

He presented the needs assessment data as well as other data 

he had assembled to back his case— truancy and vandalism^ figures , 

amount of graffiti, and the number of fights on the playground, 

He asked the staff to discuss the data and come up with a plan. 

' The resulting plan they developed was designed to improve the 
school climate. Though the principal in this case managed his ' 
role differently than the principal in the first exMiple, he 

' nortetheless guided the decision. Both of these typical examples 
from our study illustrate (1) the power and influence of the 
principal in decision-making at^he school site, and (2) some of 
,the factors that precipitate and influence these decisions. 

The situations at\the two schools illustrate the two variations 
of the model. The decision process may be|in at point A or point 
' B, ending at point C--the &ecision--in either case. That is, a 
significant program occurrence can require a decisioh which in . 
•turn necessitates- the search for appropriate data, or alternatively 
the awareness of data itself can lead to a decision^ In either 
case the principal can.be seen as the mediator who processes the 
Input Information and guides the decision.. In bpth of our examples 
the principal influenced'^ the decision greatly. Similar needs were 
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recognized in each case, but the data used, the ^timing of the 
decision, and the handling of the decisian differed considerably, 

ft = ■ . 

Regarding ^ the precipitating event, in the first case the extra 
fwiding was an external occurr&nce that led to the decision. In 
' the second case it was the data Itself that acted as a catalyst 
for the change.. These factors in turn Influenced the timins of 
the decision. The data used to make "the decision afso differed. 
At the first school an idea generated from the principal's past 
experience led to the decision. At the second school the principal 
selected data as well as needs assessment data led ta the decision. 
In terms of the handling of the decision, the principal whose 
idea from past experience was the key decision^ .factor made the ^ 
decision based on positive feedback from his staff, XKe principal 
whose data precipitated the decision allowed, the idefi to emerge 
from his staff In the process of group dynamics. In this study we 
will attempt to .account for the differences Illustrated by these 
two exampl^es, and to see what bearlcgs", if any, these differences 
have on the utilization of evaluation data, 

» 

The Findlnss * 
We have described two typical cases noted in our observations - 
that led to depicting the relationships presented in the model. 
Our" data in these examples %how that the school principal ^Is the 
mediator of the relationships between (1) A and B, the ' significant" 
occurrence and the data used to make the decision, and (2) B and 
C, the data and the decision, " . ^ 
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Although the data shows that there were a. couple of schools where 
persons other than the principal commahded de facto, authority over 
program decision-making, in most cases the prlncl.pal was the locus 
of the, decision making context. The final approval for all 
decisions' if not the decision Itself was of course always the" ' 
responsibility of the principal. Not only was he or she the final 
authority for all decision making, he or she determined to a gregt 
extent what Information should be used In making decisions, as ' 
indicated in the cases discussed above 

Individual differences in principals' orientation to their 
management or decisloh making role became apparent to us- during 
the interview process. When asked to describe his or her duties 
in relation to the specially funded programs at the school, one 
principal responded as follox^s. 

(a) "It is the principal's duty to Implement the policies 
and procedures of the board of education- the guide- 
lines for all these programs are relatively inflexible 
It IS the principal's job to see that the school is In 
compliance with these guidelines, to monitor the proBrain 
to see that they're iri compliance." (13P), j 

-fc=«oatrast with this description, another principal characterized 
his duties this way- \ ■ - 

(b) "I think the uniqueness of a school wlth^ this kind of 

funding IS that you -have an awful lot of personnel "And 
you have an awful lot of help. T/e have a billnguai 
. coordinator we nave a person out in the yard for motor 
skills, we have an ongoing daily staff development 
program where I'm able to meet with teachers for a half ' 
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hour each day by grade level (primary , middre , and upper) 
Realjy, I think, Iji. terms of meeting the guidelines , ' 
the dbjectives of all these pragrams, I see my "role as 
get ting--^developing--a CQhesive staff, and taking 
time necefesmry almost on a daily basis for ongoing 
pjlahning, ' problem solving, program development, and 
developing a high Tnorale and commitment to the program," 
(02P) 



As we gatliered quotes from the other -interviews a pattern began to 
emerge* Let us first try to characterize the distinctions we noticed 
and then describe where these characterizations led us in capturing ' 
J, their association with various types of evaluation utilization. 

As we reviewed the role responsibility data from the principals' 
interviews, We foupd others that seemed similar to quote (d) above. 

(c) It is extremely important to bring the school into 
compliance=-it is the number one riisponsibility of the 
school principal. Another major. role is organizing the 
advisory coriimlttees (compliande) that go with the two 
programs .. (06P) /Paraphrase^ 

(d) ■'Related to the Comp, Ed,^ and SI program, /my dutW^ is 
to oversee them, to make sure that the peo^e who are in, 
charge of the Individual programs are carrying out the 
policies that are set pp^-by the Federal' Government , and 

. also by the State Government. To supervise pei ioxinel, 
to bring to personnel and to the coordinator i any new 
changes ox additions to policy." (lOP) 

The three principals whose concepts of their roles are represented 

in quotes (a), (c) and (d) show concern for compliance issues, . 

indicating that the^se directly impact their duties. At the other 

end of the speetrxam are those principals who see their duties moreen 
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terms of internal school eoticerns , such as represented by quote (b) 
above. ^Ocher principals similarly Indicated an emphasis on what 
they can accomplish as an end in Itself , at the school site* 

i-. E 



(e) "To be perfectly honest, T have very little to do with. ^ ' 
thosfe programs other than the overall supervision, of the 
persons who handle those programs. I make sure that we 
are on track, monitoring that we are moving ahead." (12P) 

<f) "If 1 were to do what the district board, my iimedi^te 
supervisor, the district / 3the state of California,^ and 
the government wanted me to do-, .1 would have long "been 
gone from this earth-- long gone, • So It ^becomes a ' . 

^ case ^ of survival, and very much so. Fortunately , I'm 
able to do this., some of my peers cannot and they've had 
some very lerious problems--heart disease, what have you. 
Because in the last ten years or so, since education has 
gone from the local control to the national, the state, 
, . , the paperwork, 6he demands, the compliances, 
, all these things h^ve come our Well, thpy add things, 
and very little resources with It." (29P) 

Not only did this second group ^Qf principals not mention cotppliance 
Issues 1^ relation to their 4utles and reapomsib.ilities vis a vis the 
consolidated programs,, their tone and attitude tended to - ^ 
indicate a lack \of concern ir general for so-called bureaucratic 
matters. This is not Co s.av H4t they did not attend to those' 
matters; merely, they did not focus oh such issues in the represen- 
tation of their own duties. If the^ first gi^oup of principals quoted 
abova might be described as "compliance-oriented," we might then 
refer to the managemint style of the second group as "non-compliance' 
orientfed." = . . . ' ^ \ ^ ' 
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The question we then began to ask ourBelves was--are these quotes 
a fair representation of the administrative styles of the 

^principals? There would be no poirrt in pursuing the matter of 
utilization prediction if the data we were picking up were Invalid 
The next step was to carefully study the data suimarles for bhose 

-principals who seemed to definitely fall within one category or 
the other. . . ' ' ' ' ^, 

V 

We reviewed both their role definition statements and ^ther points 
tti the open interview where 'references to their particular admin- 
is^trattve points of view were found or implied* After identifying 
these "classic" examples we qade lists of criteria which seemed 
to fit each category* These ^criteria were =then used as a standard 
against which to compari^ the' other pririctpals wlio less clearly 
preferred one style or the other. "These criteria are sunmiarized 
below as charactterl'stl^ of the two types of principals, * ' 

WhAt we found was that there was a consistency of approach; we 
were *able to ^characterize most of the 23 principals on the basli 
of these criteria as being either "compliance" or "nori*- compliance" 
oriented. Principals represented by quotes a, Op\d^ and six ^ 
others were compliance oriented^ exhibiting thre more of the 
criteria listed, ^ose represented by^ quotes b , e , f, and eight 



Characteris ties of . Principals ' Administrative Styles 



Compliance Oriented 


.Non-Compliance Oriented 


1. Role description--f ocuses^ on 
Implementing the policies of the^ ^ 
Board of Education; sees that 
the school's prograjns are in com- 
pliance with the Title 1 Guide- 
lines' and/or the writtfin^,^^ 
proposal, ' 

2. Has trouble identifying decis- 
ions made at the school site, 
indicating ^that the mandated pro- 
grams do not allow much flexibil- 
ity* Identifies decisions that 
are beyond the control of the 
school, ^ ' 

3. Gives as reasons for decisions 
that they are program mandated* 

4. Focuses on budgetary m.atters 
and other externally-monitored 
yariables as factors leading ta 
the decision. 


1, Role description- -views the 
school as a more or less auton- 
omous unit, with its o\m capabi*. 
lities.- for decision making and 
problem solving. Focuses on 
administering , a smooth running, 
self sufficient school rather 
than on the need to comply with 
eKternal requirements* 

2* Readily identifies decisions 
that have been made locally, 

3* Gives ' internally generated 
data-based reasons for decisions. 

4* Focuses on staff inputs as 
key decision variables. 



others were .non-compliance oriented, eKhibiting at least three out 
of four of the listed criteria for that category. ' There were one 
or two principals .who did not clearly fall into either category. 
This is not to say that a clear-cut .dichotomy on the compliance y 
dimension 'is represented in principals ' administrativds styles, but 
that, general tendencies can be Identified in most cases. 



Having raanagad to categorize administrator si styles, we then 
wondered if the degree to which they exhibit one tendency or^ the 
other may predict an index of their utilization of" evaluation 
information'* Is administrators' style as characteriEed in our 
data closely aasociated with the utilization of evaluation 
information? Let's examine the kinds of evaluation utilization 
that were mentioned by principals in each of the, 6wo groups. 

First we -11 outline the kinda of data, ^oth extamally and ^ 
internally generated, that were available as perceived by those 
interviewed. Then we'll describe the patterns of use that emerged 
in our analysis.' It should be born in mind that the range of 
possibilities for external ly_gener a ted evaluation data was limited; 
poastbtlittiea baaically included the PQR (state or district) 
and test scores^ Schools coul^d request^^ecial evaluations from 
diatrict off icea but we fouhd no exampl&i in "our data of that 
occurring. As far as ittternal evaluation was concarnedV a needs 
assaaament soliciting input from staff md conmunity was required, 
but thsre were no Specif ica^ to what . form that assessment 

'should take/ Poasibilities for other kinds of internal evalua' ' 
were theoretically endless, but not necessarily required by 
axtemal authorities or preaaures. Internai evaluation activitiea 
included ,needa assessment, devaluation questionnaires, e.g. -rating 
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sh^ el¥~oF~s^^f £ development ac t i v4 tie s /^apd informal __dls cus s i ons 
at general staff meetings, grade level meetings , and team leadership 
meetings. The community had input through advisory councils and 
written needs assessments. One or two schools also had evaluation 
committees which monitored program implementation in the classrooms*' 



It turns out that despite this fairly limited range of types of 
evaluation activities, the kind of ^evaluation which tendfed to be 
utilized by a given principal can be predicted by his or her compllanqe 
prientation. Those principals we categorized as "compliance^ or left ted" 
showed a different pattern 'of evaluation utilisation than did principals 
classified as "non-compliance oriented." 



TlflB following ekamples from our data illustr'ate the kinds of data 
«>mpllance^ortanted principals utilized. 



Group A, Compliance-Oriented ^ 

(regarding the PQR) ' ' . 

"It does help for somehody else to look at you , . , some- 
body outside . * »* to tell you some of the good things you're 
doing and where you can improve, \ I think it?s, in a way, 
support', . , for somebody to come in who*s an authority,- 
.shall we .say, as well as the district, to say, 'This, we 
think, will help the children improve in reading inprcve in . 
math and language * , , * ' I see 'it as. a positive thing," (06P) 

"From' the team that came in and gave us> a mock MAR we were ' 

able to see some pf the areas that I don't see or the 
^ ^ resource people dbn't see, because sometimes it takes an 

outsider to come in and ^ee^ things that you don't see* You 
' don't always see your own faults, and so there you have to 

have outsiderg," (OSP) - 
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-(regarding test^ dafr,a)_, . _ " ' 

"T. think that we have a better understandlngi^'now, through- 
working closely with as to what the tests mean and how 
they can help in working with»the children." (09P) 



"The basic information that he ^he testing coordlnatorj 
used was the testing materials, the teachers turned into him 
based on the. test material which is pretty comprehensive. W 
do a lot of testing, ^on the DGD. And the sunmaries the 
teachers turned into 'him- -it Just seemed that we were not 
meeting what that objective was In tBe program-." (15P) 



•H^. . . it's not whatever the teacher t?fiinks . . . it's what 
the test shows." (20P) » ' . 

In nearly every zaae in our data, the Group A principal focused 
on the utilization of PQR data when the topic'' of evaluation was 
introduced. The second most frequent focus was test data of the 
standardized type. We noticed that both of these types of 
evaluation data are externally created and ln^osed on the school. 
Non-comp-liance principalff, on the other hand, focused on the 
Utilization of ^ internal evaluation activities to a far graater 
extent' than did eompllanea- oriented principals. Most of these 
non-compliance principals Indicated a^negatlve attitude toward 
externally generateji evaluation data, . 



Ins'tead non-compliance oriented principals looked to data 
genarated within the' local school as the mpst valid and relevant 
for decision making, as can be seen in the following quotes. 
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' Group B . ~ Hon - Comp 1 1 an c 0 r 1 en t e d 



(regarding Internal evaluation; meetings) ' v - . . 

"We hava a faculty: meeting every— solitary wj^ 

theory being that (ir they're the ones who know thS^prmiema, 
net I; and (2) if W carr get a problem out of the way before^_ 
it -Starts getting big and festering it's much, better . * . (26P) 



"I think evaluation, reaily--for it to transfer back into 
what's actually happening in the classroom,. I think it has to 
really involve the people that are implementing the program. 
They have to discuss whatever topic or problem long enough ^ 
where they're really involved, where they're really attemtping 
to come up with solutionis You can't solve anything unless 
you really , evaluate the /situation. They. do this, on a daily 
basis either between team partners, ^aS' a total group--it 
depends on different aspects of the progrMi," (02P) ^ 



(internal evaluation: evaluation comnittee) 

"We dp a local school /"evaluation of : our program. We do 
this every year. We set lip. our own team. And that ^^am^ 
consists of classroom teachers, parents, aides arid admlnistra- 
tors. And at that time they go through the rooms and they 
t^e a look at the various: .components of instruction.. We 
^ don*t think of it as a 'mock PQR,.' These are our own needs. 
We don't ^try to mimic that.' . This is strictly for us/" (27P) 



(interiial evaluation: observation) 

We do a lot of foOTial evaluation that really, is worthless . . * . 
What we get through TSM is a lot o£ statistics that really 
^ : have no meaning for us Evaluation for me is this ^ alright, . 
we're hbving a book falr^ and ^e worked on it last year ^ * * 
and teachers really weren't into It* what was the problem? 
They really didn't know how %p get liSto it. That was our • 
' evaluation. So what are wa going to dp? ' We're going to 
plan differently next year so that teachers will be better 
prepared to do it. They'll ha^e cjore background i they'll 
know hew to get the children mptlvated. You don't dp this 
formally. You make observations. A lot of people are maklng> 
observati-ona. Teachers cotae back, and they talk to you. We 
do a lot of talking to each other.' (19P) , V, - 
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Group B. Non-Compllarice Oriented 

(regardtog internal evaluation) 

. ."The -informatl that comes from people and agencies that 

purport to serve us Is about 99% ineff^ctlver The information 
that wf aa.t upon is generally self -generated, " (03P) 



The apparent pattern in these examples is that the group B prlnci- 
'pals tended to regard externally generated data as being of very 
limited use to them in their decision making. Terms and phrases 
such as "mindless," "almost useless" and "99% ineffective" in 
reference to PQR's and test scores do not indicate' much^ confidence 
in these measures; This group apparently prefers to make decisions 
on the basis of d^ta over which they have more control--intemally 
generated data from staff meetings , . evaluation cormaittees , needs 
assessments arid observations* - , 

What these patterns seem to^be tellitig, overall, is that principals 
who look to externally generated data for making decisions are 
those who emphasize exterrLal, compliance issues in tlfelr responses 
to questions about significant occurrences //decisions, and the 
factors that precipitated them. Conversely , the non- compliance 
ortented principals tend to see the internal structure oj! fcjie sphool 
setting as providing their frame of reference. 

StMasnary and Conclusions ^ 

We b^aan our study with the belief that an examination of some 

I " " : " " ■ " ' " ^ 

f satires noted in our data would give us additional ^wsights into 
aSmimstrative style as it Impacts on evaluation utillEation. 
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A rich iouree of data from our Interviews -with elementary school 
principals contained some interesting though dissimilar cases, 
regarding some aspects of the decision making context* Based 
on these data, we proposed a model that would account for the 
primary context dlsiimilarlties--the decision source as a signi- 
ficant occurrence or as the awareness of data; a model that 
would show the principal as the mediator of the relationships 
among the A, B, and C factors in the decision conteKti a signi-^ 
fieant occurranca, data, and a decision, EKamples dra^n from 
our data showed how we derived the model* IJavlng broadly outlined 
these relationshlpi , we focused our analysis on the principal as 
the mediator'^ lootoig f or trends in our daca which might bear on 
evaluation utilization patterns. We think we uncovered some 

tf ends , in the compliance Orientation of adtinistrative style, 

■ " . f . = -- . 

Nearly every principal could be categorized according to his or her - 
orientation to issues of compliance. These orientations turned 

out to be good predictors of use patterns. 

it - ■ ' . . 

What we have tried tOipugges.t l€ ati orientation, not a dichotomy, 
A cQmpilance-^orlerited principal who focuses on test scores does 
not necessarily eschew internal evaluation altogether i a non^ 
compliance oriented principal may find PQR recommendations less than 
valid but, does not disragar;d them totally. If jPollow-up studies 
with more Intensive conceptual and statistical analyeea shpw that 
thi^ relationships and predictions we suggest here can be supported, 
these will have triplications ^or evaluators, progrma directors 
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and principals. Principals can become more aware of their own pat 
terns and.. through a better understanding of the way their own 
style influences their use of information, can be in a better 
position to alter theie patterns. Program directors can recognise 
the need for a balance In these orientations when selecting, staff-. 
Program-evaluator^ c|n consider Indlvidual^preferences when de- 
.signing^ their evaluations. We suggest these directions for 
future studies of evaluation utilization. 
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